Optimal Coded Sampling for Temporal Super-Resolution

Conventional low frame rate cameras result in blur and/or aliasing in images while capturing fast dynamic events. Multiple low speed cameras have been used previously with staggered sampling to increase the temporal resolution. However, previous approaches are inefficient: they either use small integration time for each camera which does not provide light benefit or they use large integration time in a way that requires solving a big ill-posed linear system. We propose coded sampling that solves both the issues: using N cameras it allows N times temporal superresolution while allowing ≈ N2 times more light for each camera. In addition, it results in a well-posed linear system which can be solved independently for each frame, avoiding reconstruction artifacts and significantly reducing the computation time and memory. Our proposed sampling is optimal, since it utilizes Hadamard multiplexing to achieve the maximum possible SNR in the recovered video. We show how to implement coded sampling on off-the-shelf machine vision cameras. We also propose a new class of invertible codes that allow continuous blur in captured frames, leading to an easier hardware implementation.